Several studies report code quality issues in academia by analysing student submissions. However, most of them focus on novices or a specific integrated development. environment (IDE), and the findings might be less representative of code quality issues in general undergraduate computing. This study summarizes code quality issues from seven programming courses with various level of complexity. There are 931 assessment tasks with 15,323 Java/Python program files involved. The reported issues are specifically tailored to computing undergraduates and are selected with checkstyle (Java) and Flake8 (Python). Our study finds that students often neglect to use blank lines between code components, braces where they are optional, and a space after a comment marker. They also sometimes include too much code in one line. This initial study will be expanded via the development of a tool that can automatically summarize the code quality issues of each student submission.

When dealing with academic integrity in programming, code similarity detectors are often used to raise suspicion of student programs with undue similarity. However, it appears that only detectors designed for that purpose are discussed in computing education publications, although similarity detectors for other purposes might also be beneficial, both to foster research and to avoid 'reinventing the wheel'. This paper maps similarity detectors used to identify code clones, repeated code segments in a software system, to the classifications used for academic integrity purposes. It will help academic integrity researchers to understand code clone similarity detectors and to read papers from that area. While there are other reviews of code clone detection, ours should be easier for computing educators to read as it uses the terminologies of academic integrity.

When checking student programs for plagiarism and collusion, many similarity detectors aim to capture semantic similarity. However, they are not particularly effective for strongly directed assessments, in which the student programs are expected to be semantically similar. A detector focusing on syntactic similarity might be useful, and this paper reports its effectiveness on programming assessment tasks collected from algorithms and data structures courses in one academic semester. Our study shows that syntactic similarity detection is more effective than its semantic counterpart in strongly directed assessments, with some irregular similarity patterns being useful for raising suspicion. We also tested whether take-home assessments have higher similarity than in-class assessments, and confirmed that hypothesis. Consistency of the findings will be further validated on other courses with strongly directed assessments, and a syntactic similarity detector specifically tailored for strongly directed assessments will be proposed.

Code reuse is a practice that may both support and hinder the learning process of programming students. To help offset the negative impacts of code reuse, many educators rely on code similarity detection tools as a first step in detecting code plagiarism. An alternative approach, often applied in parallel, is to help deter plagiarism by teaching students the importance of referencing externally sourced code. However, there are no broadly accepted standards for referencing in programming. We have found one standard proposed in the literature, and in this paper we explain the design of a system to help programmers apply that standard. The system we will build is a semi-automated code comment generation system that will rely on a code clone detection approach for detecting similarities between the student's code and the code at a website whose URL the student provides. This will assist the user to create appropriate references, in the form of inline comments, when they reuse code from websites or other sources. This work in progress paper explores the relevant literature, explains the design choices of the system and the plan for its evaluation, and presents a progress report on the work.

When using code similarity detection to uncover code plagiarism and collusion, the marker needs to determine whether any detected similarities might be the result of coincidence. But understanding the similarities can be difficult and might be prone to human error, because few tools facilitate the investigation process, and if they do, the similarities are not explicitly explained in human language. This paper presents STRANGE, an investigation module that exclusively explains code similarities in natural language (English and Indonesian). For the purpose of reusability, STRANGE can be embedded in JPlag and other code similarity detection tools. It can also act as a standalone tool for measuring source code similarity. Our evaluation shows that STRANGE is more helpful than JPlag in the investigation process since it explains the similarities in natural language. Further, its effectiveness is comparable to that of JPlag but higher on trivial disguises of the sort that novice students will tend to apply when disguising copied code.

This research-to-practice work-in-progress paper presents an automated feedback tool that can be used in many teaching environments by integrating it with a web-based assessment submission system. Each time a student submits their work, they will automatically get feedback about aspects of the code quality. Automated feedback tools have been developed to educate students about code quality. However, integrating such a tool into an existing teaching environment can be challenging as these tools can depend on particular working environments, can be separate from the assessment submission system, or can require historical data. Our initial evaluation shows that the tool can be helpful as students do sometimes neglect to satisfy all code quality requirements. However, some false results are expected for spelling correction as student programs are not written in natural language. According to our quasi-experiments, the tool substantially reduces the number of word misspellings in comments due to their substantial frequency of occurrence.

Some code similarity detectors are designed to address academic integrity in early programming courses by recognising subtle variations, in the assumption that the code similarity in these courses is typically higher than that in later courses. Although the assumption is often used, it has no empirical evidence, and might be misleading. This study empirically investigates the assumption by examining the relationship between code similarity and course semester in seven programming courses with a total of 931 distinct assessment tasks. Our study shows that the argument is not necessarily true since in later courses, some assessment tasks require the students to follow a particular structure, to use external libraries, or to implement specific algorithms taught during the course.

When student programs are compared for similarity, certain segments of code are always sure to be similar. Some of these segments are boilerplate code -- public static void main String [] args and the like -- and some will be code that was provided to students as part of the assessment specification. The purpose of this working group is to explore what other code is expected to be reasonably common in student assessments, and should therefore be excluded from similarity checking. The answers will clearly vary with programming language, and perhaps with level of assessment item. Working group members will collect assessment submissions from their own or their colleagues' students, and it is hoped that these submissions will together encompass a wide variety of assessment tasks in a wide variety of programming languages. The working group aims to deliver clear guidelines as to what code can reasonably be excluded from automatic code similarity detection in various circumstances. It also aims to deliver a summary of what sort of code lecturers tend to provide for students when setting an assigned task, and why they provide that code.

This paper presents an analysis of all 1295 full papers and 129 working group reports presented and published at ITiCSE since the conference began in 1996. Working group reports are analysed separately from full papers, in recognition of the particular process by which they are created. The analysis shows that nearly 40% of ITiCSE's full papers concern programming education, and that more than half of them present work conducted in single courses. In contrast, most of the working group reports have a context that extends beyond specific topic areas, and report on work that was not conducted in any courses. About half of the full papers focus on techniques of teaching and learning or tools for teaching and learning, whereas two thirds of the working group reports focus on techniques of teaching and learning or curriculum. With both full papers and working group reports there has been a steady increase in the proportion of papers that clearly present educational research. The findings from our analysis provide valuable insights for potential authors, attendees, chairs, and the whole ITiCSE community.

Since 1996 the ITiCSE conference has provided a forum for academics with an interest in computing education scholarship and research. ITiCSE is often thought of as the European counterpart to northern America's SIGCSE Technical Symposium. Yet from its very beginning, ITiCSE has differed substantially from the Symposium. In addition to the many participants from Europe and North America, ITiCSE has also always hosted many participants from Australia and New Zealand, expanding its international perspective. Why is this? What is the appeal of ITiCSE to its participants, and what do its participants bring to ITiCSE? More specifically, what do Australians and New Zealanders bring to and take from the conference? Are there opportunities to involve computing educators from other regions? Judy Sheard and Simon will explore these questions and provide their perspectives. In this 25th year of ITiCSE, they will also speculate about the future of the conference and of the future role of Australia and New Zealand -- especially in light of the disruption in 2020.

To act with academic integrity in programming, students need to understand the concept of code similarity and the aspects that contribute to it, because undue similarity is often used as a first step in detecting plagiarism or collusion. However, if students are told at all about code similarity, it is typically in very general terms, which might not be clear to all students. This paper proposes CSD, a tool to independently educate students about code similarity through the application of semantic-preserving disguises. Given Java or Python code, CSD will disguise the code without changing its semantics. CSD can apply 60 types of disguise covering five code components: comments, white space, identifier names, data types, and constants. The tool will be integrated into a submission system, so that each time a student submits their code, they will be shown a version of the code with some disguises applied, along with a related explanation. This is expected to motivate further use of the tool while educating students about the futility of trying to copy and disguise code.

It is well documented that some students either work together on programming assessments when required to work individually (collusion) or make unauthorised use of existing code from external sources (plagiarism). One approach used in the detection of these violations of academic integrity is source code similarity detection, the automatic checking of student programs for unduly high levels of similarity. Preprocessing of source code files has the potential to increase the effectiveness, the efficiency, or both, of the source code comparison process. There are many possible steps in the preprocessing, and examination of the literature suggests that these steps are selected and implemented without any empirical evidence as to their value. This paper lists 19 preprocessing steps that have been used in code similarity detection, and assesses the effectiveness and the efficiency of 16 of these steps on data sets of student programs from introductory programming courses. The results should help researchers to decide what preprocessing steps to include when designing source code similarity detection techniques or software. According to the study, identifier removal increases both effectiveness and efficiency. Token renaming and syntax tree linearisation increase effectiveness at a cost of efficiency. Other preprocessing steps are dependent upon characteristics of the data set and should ideally be empirically tested before being applied. The paper should also help alert programming educators to the sorts of disguise that students can apply to copied programs.

Authors of academic papers are generally required to nominate several keywords that characterize the paper, but are rarely offered guidance on how to select those keywords. We analyzed the keywords in the past 15 years of selected computing education publications: the 1274 papers published in the proceedings of ICER and ITiCSE, including the ITiCSE working group reports. As well as the keywords assigned by the authors, we mined the abstracts of these papers to extract a separate list of keywords. Our work has two goals: to frame the thematic landscape of the field, using keywords that communicate the work conducted; and to detect differences between the human judgement and interpretation of keywords and the machine 'intelligence' on handling those keywords, with respect to the clusters of thematic topics identified in each case. The analysis shows that the field is dominated by learning approaches (e.g., active learning, collaborative learning), aspects of programming (e.g., debugging, misconceptions), computational thinking, feedback, and assessment, while other areas that have attracted attention include academic integrity (e.g., plagiarism) and diversity (e.g., female students, underrepresented groups). It was observed that the keywords chosen by authors are often too general to provide information about the paper (e.g., 'concerns', 'course', 'fun', 'justice'). We elaborate on the findings and begin a discussion on how authors can improve the communication of their research and make access to it more transparent.

Research into the relationship between learning computing and students' attitudes, beliefs, and emotions often builds on theoretical frameworks from the social sciences in order to understand how these factors influence, for example, students' motivation, study practices, and learning results. In this paper we explore the computing education research literature to identify new theoretical constructs that have emerged from this research. We focus on empirical work in programming education that extends or adapts theories or instruments from the social sciences or that independently develops theories specific to programming. From an initial data set of more than 3800 papers published in the years 2010--2019, we identify 50 papers that present a range of domain-specific theoretical constructs addressing emotions, affect, beliefs, attitudes, and self-efficacy. They include 11 validated instruments and a number of statistical models, but also grounded theories and pedagogical models. We summarize the main results of many of these constructs and provide references for all of them. We also investigate how these constructs have informed further research by analysing over 850 papers that cite these 50 papers. We categorize the ways that theories can inform further research, and give examples of papers in each of these categories. Our findings indicate that among these categories, instruments have been most widely used in further research, thus affirming their value in the field.

Computing education and outreach in the K-12 school sector have shown significant growth over recent decades, resulting in a large body of literature focused on the teaching and learning of computing. Despite this extensive literature, we are not aware of global overviews on teaching and learning programming as opposed to computing or computational thinking in K-12. We conducted a systematic review of the literature on introductory programming from 2003 to 2017. In this paper we review the papers that were set in the K-12 context with the aim of exploring developments that have been made in teaching introductory K-12 programming during this period. These include new programming languages, tools, teaching methods, and outreach programs. The impetus for these innovations was often a desire to provide interesting and engaging learning experiences and to ensure an appropriate level of instruction for a particular age group. Many initiatives were driven by changes to national curricula to mandate the teaching of programming. We find that there is a need for long-term studies to identify the most effective pedagogical approaches. We also identify a major need faced by many countries for training and resources to support teachers through the curriculum changes.

Vast numbers of publications in computing education begin with the premise that programming is hard to learn and hard to teach. Many papers note that failure rates in computing courses, and particularly in introductory programming courses, are higher than their institutions would like. Two distinct research projects in 2007 and 2014 concluded that average success rates in introductory programming courses world-wide were in the region of 67%, and a recent replication of the first project found an average pass rate of about 72%. The authors of those studies concluded that there was little evidence that failure rates in introductory programming were concerningly high. However, there is no absolute scale by which pass or failure rates are measured, so whether a failure rate is concerningly high will depend on what that rate is compared against. As computing is typically considered to be a STEM subject, this paper considers how pass rates for introductory programming courses compare with those for other introductory STEM courses. A comparison of this sort could prove useful in demonstrating whether the pass rates are comparatively low, and if so, how widespread such findings are. This paper is the report of an ITiCSE working group that gathered information on pass rates from several institutions to determine whether prior results can be confirmed, and conducted a detailed comparison of pass rates in introductory programming courses with pass rates in introductory courses in other STEM disciplines. The group found that pass rates in introductory programming courses appear to average about 75%; that there is some evidence that they sit at the low end of the range of pass rates in introductory STEM courses; and that pass rates both in introductory programming and in other introductory STEM courses appear to have remained fairly stable over the past five years. All of these findings must be regarded with some caution, for reasons that are explained in the paper. Despite the lack of evidence that pass rates are substantially lower than in other STEM courses, there is still scope to improve the pass rates of introductory programming courses, and future research should continue to investigate ways of improving student learning in introductory programming courses.

Vast numbers of publications in computing education begin with the premise that programming is hard to learn and hard to teach. Many papers note that failure rates in computing courses, and particularly in introductory programming courses, are higher than their institutions would like. Two highly distinct research projects have established that average success rates in introductory programming courses world-wide are in the region of 67%. However, there is little published work comparing pass rates in computing courses with those in other STEM disciplines. As institutions continually ask computing educators to justify the atypical failure rates in their courses, a thoroughly researched comparison of this sort could prove useful in demonstrating whether the phenomenon is real, and, if so, whether it extends somewhat beyond the boundaries of individual institutions. This working group will gather information on pass rates in computing courses, particularly introductory programming courses, and in courses at comparable levels in other STEM disciplines. Members of the group will be required to gather the information from their own institutions, and further data will be gathered by way of a broad survey. The data will be analysed to see whether global patterns can be established, and the group will survey the literature to gather and summarise postulated explanations for any difference between pass rates in computing and in other STEM disciplines.

A recent ITiCSE working group argued the need to explicitly inform students of the academic integrity requirements that apply when they are writing computer programs. The working group proposed a wheel-like diagram that might be used for this purpose and provided some examples to illustrate its use. Three universities in Australia have adopted the principle of informing their students about academic integrity in programming assessments, but have each taken a different approach to doing so. This paper reports on the three different approaches and explains why each was chosen.

Source code plagiarism and collusion are continuing problems in academia. To deal with these issues, lecturers are often aided by automated code similarity detection techniques or tools. Students' programs are filtered by these, and suspicious groups of programs are displayed to the markers for further investigation. As the detection techniques are many and varied, it can be demanding to choose the most suitable one for a particular teaching context. This paper summarises the mechanisms by which each of these techniques works, compiled from publications listed by Google Scholar and one or more of the ACM digital library, IEEE Xplore digital library, ScienceDirect, Scopus, and the references of already listed publications. The review is intended as a guideline for lecturers seeking to choose the most suitable technique, and for researchers who are seeking to understand the current trends and the possible research gaps in this topic.

The goal of most computing education research is to effect positive change in how computing is taught and learned. Statistical techniques are one important tool for achieving this goal. In this paper we report on an analysis of ICER papers that use inferential statistics. We present the most commonly used techniques; an overview of the techniques the ICER community has used over its first 14 years of papers, grouped according to the purpose of the technique; and a detailed analysis of three of the most commonly used techniques (t-test, chi-squared test, and Mann-Whitney-Wilcoxon). We identify common flaws in reporting and give examples of papers where statistics are reported well. In sum, the paper draws a picture of the use of inferential statistics by the ICER community. This picture is intended to help orient researchers who are new to the use of statistics in computing education research and to encourage reflection by the ICER community on how it uses statistics and how it can improve that use.