Our designs of language tests understandably focus on the endpoint: a useful test with interpretable results. We should not neglect what precedes that endpoint: the critical phases in the development of the test. We do not reflect enough on the development process, perhaps because we think that this is not worthy of research. Awareness of the stages of test design is useful, however: it pays handsome dividends when the quality of the end product is examined. The wide-ranging scholarly work of Carol Chapelle has focussed, among other issues, on two critically important components of testing language ability: assessing vocabulary knowledge and computer-assisted platforms for taking language tests. Their interaction will be highlighted here. Using a five-phase approach to language test design, this contribution sets out how the intuitive initial solution becomes the prompt to greater deliberation; how the articulated specification of what is being tested leads to greater theoretical defensibility; and how considerations of economy can be accommodated, leading to justifiable technical compromises in devising a quick test of reading levels. The initial solution associated reading ability closely with vocabulary knowledge only. The more deliberate subsequent design sought to overcome some of the limitations of that premise, by enriching the test with the addition of several further task types. The record of these design decisions indicates the rise in language assessment literacy (LAL) among the design team members, and how useful such a design record can be to enhance the quality of the eventual measure.
The appreciation we have for the work of Christo van Rensburg often derives from his insights into the origins ofAfrikaans. Perhaps it underestimates his contribution to the establishment of exciting and, on reconsideration, significant language interventions. One such intervention was the establishment ofwhat is known today as the UnitforAcademic Literacy at the University of Pretoria. This contribution explores the conceptualisation and theoretical grounding of these kinds of initiatives from the perspective of applied linguistics. It is an invitation to reflect on the nature of applied linguistics, and on the fundamental notions operative in thatfield that will allow us to conceptualise the principles of language intervention design. These principles facilitate evaluating whether the designed solution is of good or of lesser quality. In this exposition applied linguistics is defined rather as a discipline of design than as that suggested by the single discipline from which its name derives, namely linguistics. Designed language interventions are central in applied linguistics, even in the broad definitions of the fi field. However, to be able to design language policies, language courses, and language tests responsibly, the designers of these interventions have to seek theoretical grounds. The analysis presented here acknowledges that theory begins with abstraction. We abstract away from factual language objects, events, states or interactions. The fi first abstraction recognises six types of applied linguistic artefacts, operative on both the normative and factual sides of the modality of design. Thus, language curricula function as the normative requirements for factual language courses; language constructs and test specifications set the normative conditions for language assessment instruments; and institutional language policies regulate and determine the factual arrangements for language use within organisations. Without theoretical support for the design, the intervention may remain just another plan. It is noteworthy that the supporting theory does not always derive from linguistics, but from a variety of other disciplines. Design is the nucleus of what we call the technical mode of experience, and responsible design needs grounding. Thatwe fi find in examining the connections of the technical modality of design with the other modes of experience. These connections between modalities are the fi first steps towards forming basic applied linguistic concepts and ideas. Their investigation reveals fundamental analogical concepts and ideas that stretch across the particular conditions which apply to language courses, tests and policies. What is more, the focus on the technical mode as being characteristic of their design, in the sense of being the leading idea in their development, reveals that it functions not only as a modality of experience, but also as a mode of explanation. In this case, the explanation is of how it is possible to form technical concepts and ideas, and how these may be articulated. It is clear that two sets of technical insights can be distinguished: technical concepts and technical ideas. From these two starting points we can distinguish respectively the building blocks, true foundation stones of language intervention design, and their complementary ideas, the technical lodestars thatfocus our designs on the future. The founding concepts may be termed constitutive concepts. They include the concepts of technical unity, technical limitations, technical regularity, effect, differentiation, appeal, and theoretical defensibility, which are considered fi first, with accompanying examples. Thereupon we examine the direction-setting technical ideas, labelled regulative ideas since they derive from concept-transcending notions. They include the ideas of technical meaningfulness, implementability, efficiency, harmony, justice, fairness and reputability. Again, illustrations taken from the South African context are provided to elucidate these. Taken together, these principles invite the designers of language interventions to give shape to them, and to do so responsibly. The discussion concludes by identifying several shortcomings in applied linguistic interventions in South Africa, both at university and at secondary school. Despite positive developments, and a much greater awareness ofthe necessity to pay more deliberate attention to the design of language interventions, there are several remaining challenges. Developments within applied linguistics show how Afrikaans has benefited from, and is likely to continue to benefit from growing interest in the fi field. Utilising the principles ofreliability, validity, theoretical defensibility, and the guiding norms ofjustice and fairness, we may entertain the hope that we may be able to design language solutions more competently, and in time.
The COVID-19 pandemic brought about disruptions to the administration of conventional academic literacy tests, which necessitated alternative ways of identifying students in need of literacy support. In response to this disruption, an existing in-house test was identified as a potential alternative for measuring incoming students’ ability to handle the demands of academic discourse. Such an alternative was necessary for the effective identification of students needing additional academic literacy support and their placement in appropriate faculty-specific literacy courses. Although the first round of institutional piloting deemed the online version of the test appropriate for being employed for these purposes, a further administration of the test was conducted to confirm whether the traditional (pencil-and-paper) version of the test yielded similar results and whether the quality of the test is such that it is appropriate for implementation at an institutional level. This paper compares the results of these two administrations of the test and reflects on the potential value of such an in-house test to achieve and enhance the synchrony between assessment, curriculum and teaching, building on earlier work on this. Keywords: academic literacy, design principles, language assessment, student placement, validity
For students who have faced previous educational disadvantage, academic literacies are key to access and participation in higher education. Reliable placement in academic literacies courses engages these learners in appropriately pitched learning activities, ensuring optimal learning while developing their confidence and motivation. In this context, we aimed to develop a reliable assessment of students' academic literacy knowledge to ensure accurate and context-sensitive placement in one of two foundational academic literacy courses. These courses were situated in a pathway program in a regional Australian university that aimed to broaden participation in higher education for individuals from under-represented groups and educationally disadvantaged backgrounds. In this article, we describe, sequentially, the use of a five-phase applied linguistics design to develop the Academic Literacy Level Test (ALLTest). The phases represent the underlying principle of allowing pragmatic considerations to precede (but not preclude) theoretical ones, and consist, in turn, of (1) identifying the language challenge, (2) applying technical imagination and knowledge, (3) devising an initial (and iteratively subsequent) solution, (4) providing a theoretical justification, and (5) revising or finalizing a blueprint. We demonstrate how this process resulted in an organizationally efficient, context-specific, valid and reliable test of academic literacy.
In the context of policy-driven language testing for citizenship, a growing body of research examines the political justifications and ethical implications of language requirements and test use. However, virtually no studies have looked at the role that language testers play in the evolution of language requirements. Critical gaps remain in our understanding of language testers' first-hand experiences interacting with policymakers and how they perceive the use of tests in public policy. We examined these questions using an exploratory design and semi-structured interviews with 28 test executives representing 25 exam boards in 20 European countries. The interviews were transcribed and double coded in NVivo (weighted kappa = .83) using a priori and inductive coding. We used a horizontal analysis to evaluate responses by participant and a vertical analysis to identify between-case themes. Findings indicate that language testers may benefit from policy literacy to form part of policy webs wherein they can influence instrumental decisions concerning language in migration policy.
In recent years a number of discussions have sustained the debate on the def of applied linguistics, a debate which addresses both ends of the spectrum of applied linguistic work: the philosophical and the practical. This paper attempts to situate its response to such (re)considerations within an interpretative framework, considering the conception of the discipline as it has evolved over five generations. The argument of the paper is that the many and varied understandings of applied linguistic work during this period confirm the relativity of the discipline and prevent its practitioners from entertaining the illusion that, because they are involved in 'applied science', theit particular response to a language problem will provide a full and finite solution.