Advancing Automated Assessment Tools - Opportunities for Innovations in Upper-level Computing Courses: A Position Paper

Teaching large cohorts in upper-level computing courses is challenging, as providing rapid feedback and marking at scale is difficult without significant resources. Many institutions lack funds to employ a large number of skilled markers or such markers are simply not available. Automated Assessment Tools (AATs) can be used to grade and provide feedback on student work, and they support incremental learning by enabling students to make multiple submissions and to correct automatically detected mistakes. Despite their advantages, the application of AATs in certain fields is limited, especially where the nature of the assessments is complex. For example, in computer graphics (CG) courses, assessing and providing feedback requires both objective source code review and subjective visual inspection of graphical output. This makes developing AATs for these courses difficult. Although various attempts have been made, we lack an overview of the field that synthesises the findings from a disparate set of papers. In this position paper, we assert that AATs should be used more frequently for CG courses, and if carefully designed with a focus on learners' needs, are useful tools to not only reduce workload for educators, but also improve learning. We undertake a literature review focusing on AATs in CG courses to gain an understanding of what has been achieved, the challenges that exist, and which areas can be developed. With this paper we want to initiate a discussion about the challenges and opportunities of AATs in CG and other computing fields that require a diverse set of skills.