Assessment design in higher education is often approached with a focus on assessment tasks and the processes of designing them, with instructors making decisions in isolation. Drawing on assessment design, co-design and practice theory as a conceptual framing, we recast assessment design as a relational, situated practice and examine co-design as a way to come to practise assessment differently. In a business school, 102 stakeholders participated across three courses through four interconnected interventions: Connect:In workshops, weekly student reflections, academic capacity-building workshops and student testimonial videos. Using practitioner inquiry and the Theory of Practice Architectures, we analysed how sayings, doings and relatings shaped changes to assessment practices. Co-design produced more inclusive and relational assessment practices, strengthened instructor design literacies, surfaced students' lived experiences and agency, and supported shared ownership of design choices. This led to changes to assessment practice within the courses. Challenges centred on time, role clarity and negotiating divergent views, yet structured facilitation and distributed responsibilities enabled progress. The article offers three provocations that serve as implications for practice: (1) reposition assessment design as an ongoing educational practice; (2) embed relationality and distributed responsibility into assessment design and (3) institutionalise assessment design as a supported and shared professional practice.
The ongoing evolution of digital technologies, particularly Generative Artificial Intelligence, continues to shape and challenge assessment design in higher education. Given the complex and sometimes competing factors that contribute to assessment design, and the evolving digital landscape in which assessment is placed, this study examines the perspectives and priorities of five key stakeholder groups - educators, students, employers of graduates, accrediting bodies, and institutional policy-makers - regarding the defining characteristics of quality assessment. Using a mixed-methods approach, we conducted interviews, focus groups, and a national survey to extend a framework for designing quality digital assessments in business education that was originally developed using educator perspectives only. The findings highlight the importance of balancing academic integrity, feedback quality, student experience, and authenticity in assessment design to address stakeholder perspectives. They also extend the framework by including two additional design elements: purpose and technology, and by emphasising the value of dialogue about contrasting interpretations of assessment quality. The study provides a refined framework that incorporates nuanced differences in stakeholder priorities, supports educators in designing digital assessments that respond to stakeholder needs, and encourages co-design and shared accountability.
Sharing evidence-based approaches is increasingly regarded as a crucial aspect of quality learning and teaching in higher education and central to the practice of academic development. Academic developers and educators are expected to continually innovate and improve their practice to meet evolving challenges. Yet, the sector lacks systematic methods to share best practice and implement solutions tested by others. Design knowledge is often tacit, difficult to distil and codify. Design patterns offer a method to systematically capture and share tested solutions to common problems in education. While existing literature documents some aspects of pattern development processes, comparisons across design pattern projects, and examinations of key considerations in the development of such projects are scarce. Using collaborative practitioner inquiry, this study provides a contextualised account of a project in which educational design patterns are developed based on a large-scale educational innovation initiative. Synthesising our experience with the literature, we uncover five dimensions that shape the formulation of design pattern projects and outline their characteristics. We show how these dimensions support academic developers and educators in planning, executing and evaluating their own design pattern development projects, and link this to broader considerations of knowledge sharing relating to educational practice.
This research offers a novel hauntological perspective to examine the evolving nature of complex learning spaces in higher education. In times of change and disruption, we draw on Derrida's concept of hauntology - which posits that the past and future 'haunt' the present - to analyse educational experiences. Our study comprises eighteen focus groups and seven individual interviews with students and educators at a large metropolitan university business school. Collaborative and abductive coding reveals critical insights into participants' perceptions of absence and presence in their experience of traversing learning spaces. Three key themes emerge: what we bring to, how we traverse, and what we are missing from higher education. The findings advocate for more fluid, adaptive approaches that balance educator and student agency while acknowledging inherent uncertainties. By highlighting the temporal dimensions shaping learning spaces, this research proposes more nuanced and responsive educational design, within and beyond the classroom.
Purpose This paper aims to identify and evaluate resolutions to key learning and teaching challenges in very large courses that involve practical mathematics, such as foundational finance. Design/methodology/approach A design-based research approach is used across three semesters to iteratively identify practical problems within the course and then develop and evaluate resolutions to these problems. Data are collected from both students and teachers and analysed using a mixed-method approach. Findings The results indicate that key learning and teaching challenges in large foundational finance courses can be mitigated through appropriate consistency of learning materials; check-your-understanding interactive online content targeting foundational concepts in the early weeks; connection points between students and the coordinator to increase teacher presence; a sustained focus on supporting student achievement within assessments; and signposting relevance of content for the broader program and professional settings. Multiple design iterations using a co-design approach were beneficial to incrementally improve the course and consider multiple perspectives within the design process. Practical implications This paper develops a set of design principles to provide guidance to other practitioners who seek to improve their own courses. Originality/value The use of design-based research and mixed-method approaches that consider both student and teacher perspectives to examine the design of very large, foundational finance courses is novel.
With the recent and rapid transition to online teaching and learning in higher education, this explanatory study investigated the key design considerations used by business educators designing assessments suitable for online delivery. From a comprehensive literature review, we identified five key design considerations for online assessments: they must (1) assure academic integrity; (2) allow for provision of quality feedback; (3) support a positive learning experience for students; (4) assure the integrity of student information; and (5) ensure all students have an equal chance to complete the assessment successfully. An additional consideration (authenticity) was identified through a survey of educators and four focus group discussions. Our analysis confirmed that scale of delivery and resource limitations, along with institutional policies and accreditation requirements, are broader and interrelated contextual factors that influence practices and decisions about assessment design. Focus group participants also identified constraints and trade-offs they negotiated in designing, evaluating and implementing online assessments. Based on our findings we propose a framework to assist educators in best-practice decision-making about online assessment design and contribute to the discourse between educators, higher education providers and professional accreditation bodies regarding online assessment.
Higher education curricula require regular renewal. Varied expertise is needed to meet the multi-faceted challenges of curriculum development, hence the importance of collaborative approaches. Co-design is one approach to curriculum making as a relational practice, with evidence showing it can improve pedagogical quality and stakeholder involvement via a collaborative ethos that breaks through constraints of individual imagination and assumptions. The actual doing of co-design in higher education remains largely undocumented, limiting our understanding of this crucial feature of contemporary pedagogical innovation and development. This paper combines the theory of practice architectures with practitioner inquiry to explore three co-design projects within a large curriculum transformation initiative at an Australian university. Each involved a course coordinator, educational developer and learning designer working with tutors, students, industry representatives and media producers. Analysis focuses on how co-design team members connected their own actions to those of others, and how they collectively accomplished things that could not have been achieved alone. Identifying these highlights co-design as a process of learning, coming to practise differently, shaped by the site at which it unfolds, but also reshaping this wider context. This analysis sheds new conceptual light on the black box of co-design as enacted in higher education.
The growing use of generative AI tools built on large language models (LLMs) calls the sustainability of traditional assessment practices into question. Tools like OpenAI's ChatGPT can generate eloquent essays on any topic and in any language, write code in various programming languages, and ace most standardized tests, all within seconds. We conducted an international survey of educators and students in higher education to understand and compare their perspectives on the impact of Generative AI across various assessment scenarios, building on an established framework for examining the quality of online assessments along six dimensions. Across three universities, 680 students and 87 educators, who moderately use generative AI, consider essay and coding assessments to be most impacted. Educators strongly prefer assessments that are adapted to assume the use of AI and encourage critical thinking, while students' reactions are mixed, in part due to concerns about a loss of creativity. The findings show the importance of engaging educators and students in assessment reform efforts to focus on the process of learning over its outputs, alongside higher-order thinking and authentic applications.
Educators rarely have access to just-in-time feedback and guiding heuristics when designing or updating assessments in higher education. This study describes the initial development process for an automated support system for designing high-quality online assessments. We identify key elements to embed in this digital artifact to offer just-in-time support for educators to design and evaluate their online assessments. We follow a design science approach in six stages, because it simultaneously generates knowledge about the method used to develop the artifact and the design of the artifact itself. Specifically, we focus on the early stages of problem identification, solution objectives, and initial conceptual design. After reviewing multiple assessment models and frameworks, we discuss a recent framework for evaluating and designing high-quality online assessments, consisting of ten design and contextual elements. This framework underpins the proposed solution which is a digital artifact that encourages consideration of alternate forms of assessment while retaining the flexibility to operate within individual educators' design practices and contexts. We expect the proposed system to help educators and instructional designers to better understand the strengths and weaknesses of their assessments, consider alternate forms of assessment, and incorporate the system into their assessment design process.
PurposeCritically, to improve and manage online learning quality (OLQUAL), higher education providers need to regularly measure OLQUAL. Hence, a reliable measure of OLQUAL in higher education from the students' perspective is indispensable. Further, as a pioneer in examining OLQUAL outcomes in the online education context, we assert that satisfaction, trust and loyalty is a global assessment that follows the evaluation of OLQUAL. A model that delineates the perceived OLQUAL and its relationship with satisfaction, trust and loyalty are currently absent. Grounded on the cognition–affective–conation framework – this study presents the indicators of perceived OLQUAL and its influence on students' satisfaction and trust which further influences their loyalty.Design/methodology/approachTo measure the OLQUAL instrument and proposed relationships – data were collected from 232 online undergraduate and postgraduate students. The results of confirmatory factor analysis measure five dimensions of perceived OLQUAL – comprising system quality, administrative quality, educational quality, transformative quality and social quality. Further, the proposed relationships were tested using structural equation modelling.FindingsThis study has successfully measured a second-order OLQUAL model on five primary quality dimensions (i.e. systems, administrative, educational, transformative and social). The findings confirm that students' satisfaction alone does not play a mediating role; rather, satisfaction and trust play a sequential mediating role between OLQUAL and loyalty.Originality/valueOur new model provides a new tool for institutions and researchers to evaluate the quality of online education programs, as well as identify their strategy in developing and providing high-quality online learning to students.
The sudden popularity and availability of generative AI tools, such as ChatGPT that can write compelling essays on any topic, code in various programming languages, and ace standardized tests across domains, raises questions about the sustainability of traditional assessment practices. To seize this opportunity for innovation in assessment practice, we conducted a survey to understand both the educators' and students' perspectives on the issue. We measure and compare attitudes of both stakeholders across various assessment scenarios, building on an established framework for examining the quality of online assessments along six dimensions. Responses from 389 students and 36 educators across two universities indicate moderate usage of generative AI, consensus for which types of assessments are most impacted, and concerns about academic integrity. Educators prefer adapted assessments that assume AI will be used and encourage critical thinking, but students' reaction is mixed, in part due to concerns about a loss of creativity. The findings show the importance of engaging educators and students in assessment reform efforts to focus on the process of learning over its outputs, higher-order thinking, and authentic applications.
This article analyzes codesign practices within a series of innovation projects that have diverse stakeholders, initially ill-defined goals, and subjective success measures. Following a practitioner inquiry approach, we draw on practice theory to analyze data and artifacts from 68 educational codesign projects shaped through new forms of collaborative working. The article examines project management principles by analyzing codesign practices and explores the interrelationship of practice and the project site. Contributions include an enriched conception of project management ontologies and a set of principles that identify practitioner mindsets and approaches that are valuable in managing innovation projects.
Design patterns make hidden knowledge explicit and shareable. They are a tool to reflect on and communicate practical educational strategies for solving recurring problems. In this workshop we will explore and share educational design patterns for connected learning at scale that use educational technology to help students build connections with educators, peers and purpose. Participants will engage in object-based learning at the Chau Chak Wing Museum to consider and reflect on the nature of patterns and discuss how patterns are present in their educational work. After collectively identifying educational challenges related to ‘problems of scale’ in teaching and learning practice, we will use a design pattern template to discuss and capture potential technology-enabled solutions to challenges relevant to individual participants’ contexts. To support this process, existing design patterns from the University of Sydney Business School’s Connected Learning at Scale Project (CLaS) will be shared and discussed, with participant feedback contributing to the further shaping and improvement of these ‘live’ patterns.
Whilst most institutions have been using online assessment submission for many decades, the move to fully online delivery during the COVID-19 pandemic meant more traditional forms of assessment also had to move online. In this national study we captured a snapshot of online assessment usage across business disciplines from 97 survey participants from universities in Australia and identified the challenges reported by participants. We found the three most predominant forms of online assessments are written reports and essays, online exams/quizzes and live or recorded presentations. We categorised the reported challenges into 13 groups, and they include academic integrity; additional time and effort for teachers; technology access and service consistency; student preferences and expectations; and changes to feedback. We conclude with a discussion on these challenges. This study contributes not only a better understanding of the usage of online assessment in Australian business education, but also provides a benchmark with which to track and evaluate future shifts in assessment practice.
This paper reports on an early-stage educational design research project to develop scalable online delivery at a higher education institution with relatively low maturity with digital learning. This involves not only an intervention aimed at transforming the curriculum and teaching practices, but also considers the broader set of institutional services that support students and faculty. The paper introduces the substantive problem in context, reviews existing design principles in the literature on high-quality online delivery and provides an overview of the emerging intervention design. This ‘whole-of- institution’ scope is fairly novel for educational design research, and the paper closes with a reflective analysis of using educational design research for this type of project.
There is growing interest in adopting three-dimensional virtual environments within spatial design courses to use digital modelling techniques to support students' learning. One pedagogical issue that has received little attention so far is the question of how 3D virtual environments can be used to implement assessment techniques that support students' learning of spatial design expertise. This chapter seeks to assist spatial design educators who are considering digital modelling by presenting two case studies that highlight assessment practices within spatial design courses using virtual environments. Rubrics and student work samples are included. These courses both involve students creating architectural designs within virtual environments, yet contrast in several ways: learning outcomes, 3D modelling technologies, and student cohorts. The cases examine how the affordances of virtual environments for iterative, immersive, and collaborative design can enable formative and summative assessment, with both design process and the final artefact playing important roles.
3D virtual reality, including the current generation of multi-user virtual worlds, has had a long history of use in education and training, and it experienced a surge of renewed interest with the advent of Second Life in 2003. What followed shortly after were several years marked by considerable hype around the use of virtual worlds for teaching, learning and research in higher education. For the moment, uptake of the technology seems to have plateaued, with academics either maintaining the status quo and continuing to use virtual worlds as they have previously done or choosing to opt out altogether. This paper presents a brief review of the use of virtual worlds in the Australian and New Zealand higher education sector in the past and reports on its use in the sector at the present time, based on input from members of the Australian and New Zealand Virtual Worlds Working Group. It then adopts a forward-looking perspective amid the current climate of uncertainty, musing on future directions and offering suggestions for potential new applications in light of recent technological developments and innovations in the area.
Anton Bogdanovych合作论文数Western Sydney University7